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1. INTRODUCTION 

One of the main serious challenges when dealing with measurement of state variables is the large 
noise or what is called outliers. Such gross errors might be due to many reasons, starting from the field 
device malfunction to simple large noise. These abnormal signals which are usually called as outliers or 
anomalies attracting attention in various fields, such as data quality in process control |I]-[B] which affect 
as well the accuracy of the predictive models, health care [4-6] network intrusion [7J-[10] where many 
approaches have been investigated and developed to defend against network attacks, environmental monitoring 
M-05], positioning estimation [16], cloud management [17|-(19], and fault detection(20|-(22]. Hence, 
various methods have been investigated and proposed to detect outliers in different applications [23}-(25]. In 
this paper, we developed the nonlinear approach of well-established algorithm to deal with estimating state 
variables of a nonlinear system containing measures that are possibly corrupted by outliers. This estimation 
will be performed using a moving-horizon estimation approach, which has been set in the preliminary result by 
Alessandri and Awawdeh for discrete-time linear systems. 

The Kalman filter (KF) is considered to be the best estimator when quadratic estimation error needs 
to be reduced or minimized. The recurring estimator works on the principle of iterating the estimated update 
depending on the current residual to generate the new output. Checking for outliers is a crucial task to ensure 
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estimator robustness [27]; where robustness is a key factor in designing of filters particularly improving the 
kalman filter robustness in the presence of outlier is required [28]. More investigation on the development of 
statistical tests and for the purpose if identification can be found on [29]-(32]. Moving-horizon estimation 
(MHE) utilizes most recent information by minimizing least squares function which is originally presented 
by and MHE results were obtained initially for linear systems [34], and then they were extended 
for nonlinear systems and large-scale systems [36]—[40]. Based on the preliminary results by Alessandri and 
Awawdeh [26], the stability scheme is driven from [37]. The approach of this paper focuses on moving-horizon 
estimator for a nonlinear systems with measures containing outliers by adopting the worst-case cost functions, 
the moving horizon estimator stability conditions for uncertain linear systems are reported in and original 
linear version of such approach is addressed in [26]. The approach includes minimizing a set of least squares 
functions while the measurement that is possibly affected by outlier is excluded, which ensure the generating 
of a the lowest cost each time, hence propagating the the estimate to the next instant. 

The organization of the paper is as follows: in section 2 the investigation of the moving-horizon 
estimation method is discussed. Finding of stability and robustness are briefed, in sections 3. Respectively, in 
sections 4 to 6 simulation results are described and discussed. Finally, the conclusion is drawn in section 7. 


2. NONLINEAR MOVING-HORIZON ESTIMATION APPROACH 

In this section, and following the success of the linear approach of moving horizon estimation in 
solving the problem of outliers in industrial systems, we propose the the nonlinear framework of such developed 
approach. Considering a nonlinear system, with t = 0,1,... 


Tiy = flee, ue) +E (la) 


y = h(t) +n (1b) 


Where x; € X C R” is the continuous state, u, € U C R is the control vector, y, € Y C R? 
is the output, & € W C R” is the disturbance of system, and 7, € V C R? is disturbance of measurement, 
assuming é; and 7, as unknown deterministic variables. Since, in this paper we are introducing the nonlinear 
theme of our preliminary works mentioned earlier; so we refer the reader to [26], and and reference therein 
for a complete understanding of the assumption and pre-definition. 

The proposed approach (see Figure 1) can be summarize as for a given measurement batch t = 
1,2,3,... N + 1, a cost function J; is to be minimized with the constrains. 


ĉitie = f (itu), t=t—-N,...,t-1 2) 


The cost function is defined for two possible cases (assumption), as follow: 


— Case 1: measurement is contaminated by an outlier (k — thmeasure), the function J; which leaves out 
the contaminated measure is defined as: 


R R - 1 : 
JE (êN) = u |lêtnae— zn |? + N 5 (yi — hên)? (3) 


— Case 2: measurement is free of outlier, the function J; is : 


(yi — h(#i,e))? (4) 
ix=t—N 


A n 7 1 
Rene) = ulne- Fowl + 7G 


Where the tuning parameter u > 0 and JF is minimized under (2). The approach is summarized as solving a 
problem as follow. 
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min Ji 
TER” s.t. {2} holds 
The estimate of x+- y at time t is: 


ĉtt- Nit = arg min Ji (2) 
TER?” s.t. B] holds 


and accordingly, 


êf yE argmin J¥ (x) 
LER” s.t. (2) holds 


k(t) € argmin Jf (êf y) 
k=0,1,...,N+1 
In case, X is known, the cost can be minimized on X, i.e., ĉ € X. By comparing case 1 and case 2, the 
optimal cost is propagated for t + 1,t + 2,.... the estimate. Recalling that, a solution exists by assuming that 
X is compact and f(.,u), h(.) and J;(.) are continuous. The sketch given in Figure |1} illustrate the approach 
of leave-one-out MHE strategy for nonlinear systems. 


Prediction 


Tt-N]t 


Yt-N+1 | Vi-n | (u-n 


Yt-1 | | Yt-1 ) | Yt-1 | | Yt-1 | 
Yt Lv J | Yt j 


Y Optimization 
Ji = Tt_Nit 


| Geva venan ) -C 


JE, m ĉi N 


a i 


Figure 1. Scheme of MHE approach for nonlinear systems when moving from t to t + 1 


With reference to and [43], (3) and (4) are considered to compare with k(t*). Concerning the 
extension of the proposed approach to the case with multiple outliers in the batch, the assumption of N + 1 can 
adjusted with a more general setting. If k outliers affect the N + 1 measurements, we consider all permutations 
of k measurements taken from the set of the N + 1 ones in the batch. Thus, for example, we need to consider 


a number of: 
B 2 
"=N 4i 


Moving-horizon estimation approach for nonlinear systems with ... (Moath Awawdeh) 


222 o ISSN: 1693-6930 


Costs to discriminate among the various cases with two outliers. Of course, we need to consider the 
“no outlier” setting and all the case that corresponds to “one outlier” in the batch. In general, we have to 
account for up to k outliers in the batch by using: 
i 
N+1 


k 
0,1,...,4 = 5 
i=0 
Costs to perform the comparisons, considering the estimate of 7;_ y at t. The stability of estimation 
error is investigated, supposing to perform a perfect or approximate minimization with fact that error is bounded 
B7]. In next section, a brief of estimation error stability with necessary recall of assumptions which were 
investigated in the preliminary work and original assumption proposed by Alessandri et al. [37]. 


3. ESTIMATION ERROR - RECALL OF STABILITY 
With the assumption that X, U, W, and V are compact set (introduced in section 2), assume that 
f(.,u) and h(.) are C? on X. 


Yi_wle = Hx(tr-w) + De, (ten )&on le + myle 


where: 


ho ft- o.o fMt-N (xiy) 
ho fit? o-o FY (2; N) 


t—1 
Hy (21-n, ty) ‘= : (5) 
ho f"=5 (xN) 
h (zi-N) 
t- 
Et- N 
EON gto Nt1 
Dh fia) Pho f(s) Sa 0 
Dg (tee) = Ob O&t—N+1 (6) 
gh A etch 
ho finy Oho fin) dhof (ny 
O&t—N Et- N41 IPE 


Following the assumption that; while the system is X observable, there exists a K-function ¢(-), such that: 
g (|e! — 2") < |En (2',u) — Hy (2",u)| (7) 


for all that x’ € X, x” € X, and u € U. Moreover, 


g (Jx _ af!) 
“ae 


= >0 (8) 


n 
Lel EX TEE" jæ =E 


under perfect minimization/approximate minimization with bounded error, if u > 0 is chosen such that: 
ae GR (9) 
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where Ly is the lipschitz constant (f, X). 
The estimation error is exponentially bounded and the optimal cost can be: 


wai luin — Ho (ĉi n)|| if k*=0 


(10) 
$ |ui nlk* — Hae (@—w) ||’ if k* € {1,...,N +1} 


Ji (ĉn) = H || ĉn — Zn ||? + 


4. SIMULATION EXAMPLE 

In this section we prove the approach robustness and efficiency by the meaning of simulation, consid- 
ering an undamped oscillator with a pulsation equal to w, the damping coefficient ¢ is assumed to be unknown 
and a constant. 


x3(t) =¢ 


The discrete nonlinear system and linear observation equations are: 


re = xÉ + Tx), 
ait) =w Pet + (1 — 2wTri jei + 12T + TE, 
Ye = 2] +m 


where T > 0 is the sample time. 

Specifically, we choose ¢ = 0.2, w = 5 rad/s, and T = 0.01 s. The distributions of initial state, system 
and measurement noises were taken according to (add Kalman book reference, example 5.3, page 173), i.e., 
they are zero-mean white Gaussian processes with covariances Po), = diag(2, 2,2), Q = diag(0, 0, 4.47), 
and r = 0.01 except in case of outliers, for which the covariance was chosen much larger than r, i.e., equal to 
10. 

Outliers were generated randomly with dispersion 0 = 10, and randomly-positioned over 100 time 
steps. We will evaluate the performances of such filters by the root mean square error (RMSE): 


ie ey 
RMSE(t) = (>. bel) 
i=l M 


where ez; is the estimation error at time t in the 7-th simulation run, and M is the number of simulation runs. 
The results of such tests with a MHF for different u and a KF for different choices of threshold values 
c, are shown in Figures 3, 5, and 7. RMSE means of the MHF and KF for different choices of u, c+, and r are 
shown in Figures 4, 6, and 8. In Table 5, the computational times for both estimators are reported. Though a 
convenient choice of u makes the MHF performs better in term of RMSE, its computational effort is larger. 


5. SIMULATION DISCUSSION 

In the mentioned example we have tried to pick up the possible best parameter tuning for KF, extended 
Kalman filter (EKF), and MHF from different simulation setting for KF/EKF choosing o+ equal to VS; or 
5/'S; or 10\/S;, and a MHF with u equal to 0.1 or 0.5 or 1, in order to reduce the effect of outliers and get the 
minimum estimation error. More test result with a MHF with u = 0.6 and EKF with no threshold check are 
shown in Figure 9. It is clear that pure EKF is not able to resist outliers. In Figure 10 a tuning threshold value 
of KF has been adopted with op = 10./S; which it has been chosen among different simulation test. RMSES 
of MHF and EKF for 100 runs with r = 0.01 and different parameters ( u and o+) are shown in Figure 11 for 
£1, £2 and x3. Tables 2 to 4 show the RMSE means of the MHF and KF for different choices of ju, ot, and r. 
In Table 5, the computational times for both estimators are reported. However, it is more demanding from the 
computational point of view as compared with the extended Kalman filter. 
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6. SIMULATION-BASED DISCUSISON ON PREDICTIVE MOVING HORIZON ESTIMATION 
(TO LINEAR-BASED) 

The proposed approach in this paper considers some complexity in computing the the different cost 
functions, obtained by excluding one measurement each time and some how computationally demanding. In 
this section we tried to consider an attempt to check for y++1 if it is outlier by computing only the minima of 
two cost functions (one computed by neglecting y;4 1 and one by accounting y;+1). However, this approach is 
much less computationally demanding but without stability proof in general (the proof is based on the precise 
requirement that the estimation is associated with the best of the optimal). This is a serious drawback, as without 
such a property it seems difficult to prove robustness, which is satisfied in the proposed method. However, 
we may guess that both stability and hence robustness may be ensured in practice and somehow evaluated 
via simulations also to measure the expected computational savings. In this section will denote the attempt 
proposed as PMHF (predictive MHF), as it relies on the prediction on the outlier occurrence when proceeding 
with the new measure update. Simulation were performed with absolute value of the outliers over the rejection 
thresholds in such a way to ensure the better conditions of work for the PMHF (the modulus of the outliers are 
randomly chosen in the range (7,,, 107,,). A simulation run is depicted in Figure 2, while Table 1 summarizes 
the results obtained over 100 runs with initial states that were randomly generated with mean equal to (1 1)" 
and covariance P) = diag(2,2), and system and measurement noises given by zero-mean white Gaussian 
processes with covariances Q = diag(1,1) and r = 0.1, respectively. For MHF and PMHF, we set p = 0.001. 

The computational effort of the PMHF is about 35% less than that of the MHF, but of course this is 
paid with an increase of the RMSEs. In any case, also the PMHF performs much better than the KF in terms of 
precision because of the large RMSEs given with all the various thresholds. Summing up, at the moment the 
PMHE should be ranked as a “heuristic” approach beacause of the lack of guaranteed stability and robustness 
properties but, since it allows on to trade between computational burden and precision, it may be the subject of 
future investigation aimed at proving such properties. 


60 nn a es = mt i T T 60 ETE as iT TTE 1 T T 

2 $ : = = = state variable z$ ae a zi = = = state variable 
GELRE measure $ $ 3 1v measure 
— MHF estimate ae Ze i > ets Í | —— MHF estimate 
=e PMHF estimate 23 = ž 3 a =O PMHF estimate 
ital KF estimate -| M P Ips EF == 1m i m KF estimate 
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time time 


Figure 2. True state, measures, and estimates of xı and x2 in a simulation run with zero-mean Gaussian 
measurement noises having r = 0.1 and using MHF and PMHF with p = 107°? and N = 3, anda KF with 


o =2S% 


Table 1. MCT (in s) and RMSEs over 100 runs for MHF, PMHF, and KF 


MHF PMHF 
N=3 N=4 N=5 N=6 N=3 N=4 N=5 N=6 
MCT 0.0273 0.0315 0.0380 0.0434 0.0209 0.0225 0.0261 0.0251 


RMSE (xı) 1.7120 1.5839 1.6698 1.7409 2.2165 2.1357 2.1965 2.2819 
RMSE (x2) 1.8337 1.7714 1.7985 1.8405 2.0471 1.9577 1.9318 1.9578 


KF 
or=VS8t or =2 fst ot =5S8t ot = 10 Vst 
MCT 0.0070 0.0067 0.0066 0.0086 
RMSE (z1) 6.9472 6.4459 9.1978 18.666 
RMSE (x2) 3.5151 3.2509 3.6555 6.6224 


TELKOMNIKA Telecommun Comput El Control, Vol. 22, No. 1, February 2024: 219-231 


TELKOMNIKA Telecommun Comput El Control o 225 
Tı 
2m T = =I L 
i i emee KF = MHF = = = True State: -© - Measure 
iti pI E a | 
ii 


= 


e~ T teehee? 
a 5 : : = 
5 | | | | tig | bo | | 

0 10 20 30 40 50 60 70 80 90 100 

Time 
T2 T3 
8 T T T T T T T T T 10 T T T T T T T T T 
= = -true state ---- KF — MHF — KF - - -MHF 

al J 


0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 
Time Time 
Figure 3. States estimation for MHF with u = 0.1 and KF without residual check 
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Figure 4. RMSES of MHF and KF for 100 runs with u = 0.1 


Table 2. Means of RMSEs for MHF with u = 0.1 and KF without residual check 


MHE KF 
if Ly x2 x3 zı x2 x3 
0.01 0.1678 0.6852 2.0905 1.5170 4.3395 2.3006 
0.10 0.2326 0.8642 2.0455 1.8418 3.3232 2.2107 
1.00 0.4818 1.4177 2.0526 1.7744 3.8066 2.1922 
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Figure 5. States estimation for MHF with u = 0.5 and KF with o; = v St 
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Figure 6. RMSES of MHF and KF for 100 runs with r = 0.01 


Table 3. Means of RMSEs for MHF with w = 0.5 and KF with o; = VS; 
MHE KF 
Ly T2 x3 T1 T2 T3 
0.01 0.1760 0.7584 2.0944 0.7144 3.4064 2.1153 
0.10 0.2124 0.7262 2.0049 0.3343 1.6533 2.0343 
1.00 0.5125 1.6230 2.0024 0.6088 2.8931 2.0204 


r 
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Figure 7. States estimation for MHF with u = 1.0 and KF with o, = 10y S4 
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Figure 8. RMSES of MHF and KF for 100 runs with r = 0.01 


Table 4. Means of RMSEs for MHF with u = 1.0 and KF with o, = 10y S; 


MHE KF 
Tı T2 T3 Tı T2 T3 
0.01 0.1742 0.7840 2.0577 0.4608 2.4653 2.1062 
0.10 0.2198 0.7199 2.0099 0.4423 1.6763 2.0488 
1.00 0.4702 1.3058 2.0794 0.8392 3.1265 2.1289 
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Figure 9. Simulation run over 100 monte-carlo with multiple randomly-positioned outliers, MHF with 
H = 0.6 and pure EKF (no residual check) 
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Figure 10. Simulation run over 100 monte-carlo with multiple randomly-positioned outliers, MHF with 
u = 0.1 and KF with residual check of o} = 10 v S4 
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Figure 11. RMSES of MHF and EKF for 100 runs with r = 0.01 and different parameters ( u and o;) 


Table 5. Means of computational time in seconds over 100 runs 


MHE EKF 

T u=01 p=06 p=l =V% of =5V5% of =10VH% 
0.01 64228 5.6620 6.2146 0.0156 0.0312 0.0112 
0.1 7.5228 5.2210 6.3697 0.0067 0.0100 0.0075 
1.0 7.5520 5.6076 6.2595 0.0083 0.0089 0.0103 
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7. CONCLUSION 


We have presented the problem of measurements contaminated by outliers in nonlinear systems using 
moving horizon estimation approach. The estimation consider two cases, measurement affected by outlier, and 
the other case where we suppose no outlier in the batch. The stability of the proposed moving horizon estimator 
is discussed. The efficiency of such filter is discussed compared with the EKF and KF. Such KF/EKF turns 
out to be sensitive to the choice of the threshold, while the proposed filter approach is easier to be tuned via 
the selection of u and more robust to outliers. The performance of the proposed method is illustrated by the 
mean of simulation and RMSE result. One the challenges in such approach is the number of outlier presence in 
the same window size as well as the challenge of computational complexity; which are considered for further 
investigation in the future. 
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